With the evergrowing sizes of pre-trained models (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting. To adapt PTMs with model parameters frozen, most current approaches focus on the input side, seeking for powerful prompts to stimulate models for correct answers. However, we argue that input-side adaptation could be arduous due to the lack of gradient signals and they usually require thousands of API queries, resulting in high computation and time costs. In light of this, we present Decoder Tuning (DecT), which in contrast optimizes task-specific decoder networks on the output side. Specifically, DecT first extracts prompt-stimulated output scores for initial predictions. On top of that, we train an additional decoder network on the output representations to incorporate posterior data knowledge. By gradient-based optimization, DecT can be trained within several seconds and requires only one PTM query per sample. Empirically, we conduct extensive natural language understanding experiments and show that DecT significantly outperforms state-of-the-art algorithms with a $10^3\times$ speed-up.

Sharpness-Aware Minimization (SAM) is a highly effective regularization technique for improving the generalization of deep neural networks for various settings. However, the underlying working of SAM remains elusive because of various intriguing approximations in the theoretical characterizations. SAM intends to penalize a notion of sharpness of the model but implements a computationally efficient variant; moreover, a third notion of sharpness was used for proving generalization guarantees. The subtle differences in these notions of sharpness can indeed lead to significantly different empirical results. This paper rigorously nails down the exact sharpness notion that SAM regularizes and clarifies the underlying mechanism. We also show that the two steps of approximations in the original motivation of SAM individually lead to inaccurate local conclusions, but their combination accidentally reveals the correct effect, when full-batch gradients are applied. Furthermore, we also prove that the stochastic version of SAM in fact regularizes the third notion of sharpness mentioned above, which is most likely to be the preferred notion for practical performance. The key mechanism behind this intriguing phenomenon is the alignment between the gradient and the top eigenvector of Hessian when SAM is applied.

Conceptual knowledge is fundamental to human cognition and knowledge bases. However, existing knowledge probing works only focus on evaluating factual knowledge of pre-trained language models (PLMs) and ignore conceptual knowledge. Since conceptual knowledge often appears as implicit commonsense behind texts, designing probes for conceptual knowledge is hard. Inspired by knowledge representation schemata, we comprehensively evaluate conceptual knowledge of PLMs by designing three tasks to probe whether PLMs organize entities by conceptual similarities, learn conceptual properties, and conceptualize entities in contexts, respectively. For the tasks, we collect and annotate 24k data instances covering 393 concepts, which is COPEN, a COnceptual knowledge Probing bENchmark. Extensive experiments on different sizes and types of PLMs show that existing PLMs systematically lack conceptual knowledge and suffer from various spurious correlations. We believe this is a critical bottleneck for realizing human-like cognition in PLMs. COPEN and our codes are publicly released at https://github.com/THU-KEG/COPEN.

Since the recent success of Vision Transformers (ViTs), explorations toward transformer-style architectures have triggered the resurgence of modern ConvNets. In this work, we explore the representation ability of DNNs through the lens of interaction complexities. We empirically show that interaction complexity is an overlooked but essential indicator for visual recognition. Accordingly, a new family of efficient ConvNets, named MogaNet, is presented to pursue informative context mining in pure ConvNet-based models, with preferable complexity-performance trade-offs. In MogaNet, interactions across multiple complexities are facilitated and contextualized by leveraging two specially designed aggregation blocks in both spatial and channel interaction spaces. Extensive studies are conducted on ImageNet classification, COCO object detection, and ADE20K semantic segmentation tasks. The results demonstrate that our MogaNet establishes new state-of-the-art over other popular methods in mainstream scenarios and all model scales. Typically, the lightweight MogaNet-T achieves 80.0\% top-1 accuracy with only 1.44G FLOPs using a refined training setup on ImageNet-1K, surpassing ParC-Net-S by 1.4\% accuracy but saving 59\% (2.04G) FLOPs.

模块化设计是未来大型空间设施的On On On构造技术的基础。标准界面是未来空间机器人系统和空间设施模块化设计的关键技术。本文介绍了Petlock的设计和测试，标准和测试无性别界面可以在未来的模块化空间机器人操纵器和航天器之间传递机械载荷，功率和数据。Petlock采用完全无性别的设计，包括连接面，锁定机制，数据和功率接口。连接表面提供了较大的翻译和旋转错位耐受性，由于其120度对称和3D形状的设计。锁定机制具有三个锁定引脚撤回结构设计，这是简单可靠的。高锁定力，高容忍度，高可靠性和低成本的优势，Petloc K在未来的轨道施工任务中具有很大的应用潜力。

向前和向后触及逆运动学（FABRIK）是一种启发式逆运动求解器，逐渐应用于具有快速收敛和生成更真实配置的优势的操纵器。但是，在高误差限制下，Fabrik表现出不稳定的收敛行为，这对于操纵器的实时运动计划是不满意的。在本文中，提出了一种结合Fabrik和顺序二次编程（SQP）算法的新型逆运动学算法，其中Fabrik推迟的关节角度将被视为SQP算法的初始种子，以避免粘在局部最小值中。通过实验评估合并的算法，在高误差约束下，我们的算法比FabRik获得更高的成功率和更快的解决方案时间。此外，联合算法可以在路径跟踪中为UR5和KUKA LBR IIWA 14 R820操纵器生成连续轨迹，而无姿势误差和最终效应器的允许位置误差。

我们认为，从其嘈杂的瞬间信息中，在任何维度上学习$ k $ spike混合物的稀疏力矩问题。我们使用运输距离来测量学习混合物的准确性。先前的算法要么假设某些分离假设，使用更多的恢复力矩，要么在（超级）指数时间内运行。我们针对一维问题的算法（也称为稀疏Hausdorff Moment问题）是经典Prony方法的强大版本，我们的贡献主要在于分析。我们比以前的工作进行了全球和更严格的分析（分析了Prony方法的中间结果的扰动）。有用的技术成分是由Vandermonde矩阵定义的线性系统与Schur多项式之间的连接，这使我们能够提供独立于分离的紧密扰动，并且在其他情况下可能很有用。为了解决高维问题，我们首先通过将1维算法和分析扩展到复数来解决二维问题。我们针对高维情况的算法通过将混合物的1-D投影与随机矢量和一组混合物的一组2D投影来确定每个尖峰的坐标。我们的结果在学习主题模型和高斯混合物中有应用，这意味着改善了样本复杂性结果或在先前的工作中运行时间。

在这个有联系的社会中，CCTV通过记录非法活动以供当局采取行动，在执行公民的安全和保障方面发挥了关键作用。在智能的城市环境中，使用深层卷积神经网络（DCNN）来检测CCTV视频中的暴力和武器暴力行为，将通过确保全天候实时检测来提供额外的安全性。在这项工作中，我们通过收集武器化和非武器暴力以及YouTube的非暴力视频的真正CCTV镜头，引入了一个新的专业数据集。我们还提出了一种新颖的方法，将连续的视频框架合并为单个显着图像，然后将成为DCNN的输入。多个DCNN架构的结果证明了我们方法的有效性99 \％。我们还通过几个参数权衡考虑了我们方法的效率，以确保智能城市的可持续性。

知识蒸馏（KD）将知识从高容量的教师网络转移到加强较小的学生。现有方法着重于发掘知识的提示，并将整个知识转移给学生。但是，由于知识在不同的学习阶段显示出对学生的价值观，因此出现了知识冗余。在本文中，我们提出了知识冷凝蒸馏（KCD）。具体而言，每个样本上的知识价值是动态估计的，基于期望最大化（EM）框架的迭代性凝结，从老师那里划定了一个紧凑的知识，以指导学生学习。我们的方法很容易建立在现成的KD方法之上，没有额外的培训参数和可忽略不计的计算开销。因此，它为KD提出了一种新的观点，在该观点中，积极地识别教师知识的学生可以学会更有效，有效地学习。对标准基准测试的实验表明，提出的KCD可以很好地提高学生模型的性能，甚至更高的蒸馏效率。代码可在https://github.com/dzy3/kcd上找到。

实体链接旨在将模棱两可的提及与知识库中的相应实体联系起来，这对于各种下游应用程序是重要的，例如知识库完成，问题答案和信息提取。尽管已经致力于这项任务，但这些研究中的大多数遵循以下假设，即可以使用大规模标记的数据。但是，当由于劳动密集型注释工作而导致的标记数据不足以针对特定领域时，现有算法的性能将遭受无法忍受的下降。在本文中，我们努力解决了几个弹药实体链接的问题，这只需要最少的标记数据，并且在实际情况下更为实用。具体而言，我们首先提出了一种新颖的弱监督策略，以基于提及的重写生成非平凡的合成实体对。由于合成数据的质量对有效的模型训练有关键的影响，因此我们进一步设计了一种元学习机制，以自动为每个合成实体对分配不同的权重。通过这种方式，我们可以深刻利用丰富而宝贵的语义信息，从而在几个射击设置下得出训练有素的实体链接模型。现实世界数据集上的实验表明，所提出的方法可以广泛改善最新的几杆实体链接模型，并在只有少量标记的数据可用时实现令人印象深刻的性能。此外，我们还展示了模型可传递性的出色能力。